Search for: All records

The Survey of Physics Reasoning on Uncertainty Concepts in Experiments (SPRUCE) was designed to measure students’ proficiency with measurement uncertainty concepts and practices across ten different assessment objectives to help facilitate the improvement of laboratory instruction focused on this important topic. To ensure the reliability and validity of this assessment, we conducted a comprehensive statistical analysis using classical test theory. This analysis includes an evaluation of the test as a whole, as well as an in-depth examination of individual items and assessment objectives. We make use of a previously reported on scoring scheme involving pairing items with assessment objectives, creating a new unit for statistical analysis referred to as a “couplet.” The findings from our analysis provide evidence for the reliability and validity of SPRUCE as an assessment tool for undergraduate physics labs. This increases both instructors’ and researchers’ confidence in using SPRUCE for measuring students’ proficiency with measurement uncertainty concepts and practices to ultimately improve laboratory instruction. Additionally, our results using couplets and assessment objectives demonstrate how these can be used with traditional classic test theory analysis. Published by the American Physical Society2024 

Concepts and practices surrounding measurement uncertainty are vital knowledge for physicists and are often emphasized in undergraduate physics laboratory courses. We have previously developed a research-based assessment instrument—the Survey of Physics Reasoning on Uncertainty Concepts in Experiments (SPRUCE)—to examine student proficiency with measurement uncertainty along a variety of axes, including sources of uncertainty, handling of uncertainty, and distributions and repeated measurements. We present here initial results from the assessment representing over 1500 students from 20 institutions. We analyze students’ performance pre- and postinstruction in lab courses and examine how instruction impacts students with different majors and gender. We find that students typically excel in certain areas, such as reporting the mean of a distribution as their result, while they struggle in other areas, such as comparing measurements with uncertainty and correctly propagating errors using formulas. Additionally, we find that the importance that an instructor places in certain areas of measurement uncertainty is uncorrelated with student performance in those areas. Published by the American Physical Society2024 

Recent years have seen a movement within the research-based assessment development community towards item formats that go beyond simple multiple-choice formats. Some have moved towards free-response questions, particularly at the upper-division level; however, free-response items have the constraint that they must be scored by hand. To avoid this limitation, some assessment developers have moved toward formats that maintain the closed-response format, while still providing more nuanced insight into student reasoning. One such format is known as coupled, multiple response (CMR). This format pairs multiple-choice and multiple-response formats to allow students to both commit to an answer in addition to selecting options that correspond with their reasoning. In addition to being machine-scorable, this format allows for more nuanced scoring than simple right or wrong. However, such nuanced scoring presents a potential challenge with respect to utilizing certain testing theories to construct validity arguments for the assessment. In particular, Item Response Theory (IRT) models often assume dichotomously scored items. While polytomous IRT models do exist, each brings with it certain constraints and limitations. Here, we will explore multiple IRT models and scoring schema using data from an existing CMR test, with the goal of providing guidance and insight for possible methods for simultaneously leveraging the affordances of both the CMR format and IRT models in the context of constructing validity arguments for research-based assessments. 

Significant focus in the PER community has been paid to student reasoning in undergraduate quantum mechanics. However, these same topics have remained largely unexplored in the context of emerging interdisciplinary quantum information science (QIS) courses. We conducted 15 exploratory think-aloud interviews with students in an upper-division quantum computing course at a large R1 university cross-listed in the physics and computer science departments. Focusing on responses to one particular problem, we identify two notably consistent problem-solving strategies across students in the context of a particular interview prompt, which we term Naive Measurement Probabilities (NMP) and Virtual Quantum Computer (VQC), respectively. Operating from a resources framework, we interpret these strategies as choices of coherent (and potentially mutually-generative) sets of resources to employ and available actions to perform. 

Disability is an often-overlooked aspect of diversity. Recent research has indicated that there are barriers to access and participation for disabled students inherent in the design of physics courses. To help counteract these barriers, universities are required to provide reasonable accommodations for disabled students. However, not all students use the accommodations they have access to because of social factors (e.g., disability stigma), and others do not have access to the professional diagnosis often required to access accommodations. The purpose of this study was to explore the experiences of students who identify with a disability/impairment who were taking an emergency remote teaching (ERT) physics course in Fall 2020 to inform policies about providing access to students in future remote and face-to-face courses. In this paper, we present the prevalence and types of impairments disabled students in physics courses reported, their reported accommodation usage, and ethical considerations of this work. Overall, we find that disabled students represent a sizeable group in physics courses, and there are positive and negative reasons students did not use or request accommodations.